Implementation - Deliverable 7

CMSC408 - Spring 2025 - Team Open Valve

OpenValve is a Steam app database. Through a combination of web-scraping and API calling we’ve curated a database of some steam users, their games, and all the tags associated with those games.

• [Link to repository]
• [Direct link to video pitch]
• [Direct link to design video]
• [Direct link to implementation video]

Design video

Open Valve

OpenValve is the database (408) assignment of Trevor Corcoran and Jon Rutan. We’re avid Steam users and owe part of our motivation in the tech field to the platform.

Goal

OpenValve is a database meant to connect Users, Games, and Tags with the purpose of finding game recommendations. By utilizing a combination of Steam Dev API calls and some web scraping techniques we’ve built a database of relevant games, users, and tags. Tags are especially important for their usage in categorizing games, and the players who trend towards certain tags in their games.

Target Audience

Us! - That is to say, all of us Steam users. As lifelong Steam users we’ve recognized it’s distinct lack of a reliable game recommendation system that is so commonplace in modern internet storefronts. So we’ve come up with our own solution, and are putting our Python and SQL mettle to the test by designing this system.

Database Design

erDiagram 
    
    GAME {
        int appID PK
        string title
        string description
        date release_date
    }

    TAG {
      int tag_id PK
      string name
    }

    GAMETAG {
      int game_tag_id PK
      int game_id FK
      int tag_ID FK
    }

    USER {
      int user_id PK
      string name
    }
   
    LIBRARY {
      int library_id PK
      int user_id FK
      string appID FK
    }

    GAMETAG }o--|{ TAG : has
    GAME ||--|| GAMETAG : has
    USER ||..|| LIBRARY : owns
    LIBRARY }o..o{ GAME : contains

Here is a diagram of our database layout. Users have a Library, in that Library they have many games, and games have many tags. Library is a join table between users and games, and GameTag is a join table between games and tags.

Gathering Data

To gather our data we use two approaches:
- Steam API
- Web Scraping

We adhered to Steams acceptable use policy in regards to web scraping and rate limited our own program so as not to cause any trouble from our end or Steams end.

Steam API

We utilized the Steamworks Dev API to gather information on users. This was primarily used for grabbing a users library to extract all the games out of it.

Web Scraping

The primary tool for data gathering was our webscraper, scrape.py. This uses the BeautifulSoup4 library to scrape the data from the steam store pages. It’s automated based on ranges of AppIDs given to it. This is used in conjunction with import_game.py which initiates a SQLAlchemy connection to the database to import the games and link all the tags that appear on the store page.

Reflection

How was your overall experience with this semester-long project?

It was great. We had a lot of fun both planning out our project and in the implementation itself. Being able to choose our topic gave us a lot of freedom in the way we approached gathering our data. It gave us both a good opportunity to do some webscraping, API calling, API development, and many other very useful techniques and tools that will serve us well in the future.

How did the homeworks and discussions prepare you (or not) for successful completion of this project?

The homeworks were quite useful, getting exposure to tools like the SQL connection, SQLAlchemy, Poetry, Docker, and mores. These tools are extremely valuable in work environments, and we were happy to implement them into the project. You can see in this repo we use all of the ones we covered in class.

What advice would you give to students working on this project in the future?

Pick something you like. We like games so it just made sense to make a database on them. I could imagine the slog of gathering and organzing arbitrary, mundane data may destroy ones motivation to pursue this project. Picking something we had an interest in, and that actually provides functionality outside of the domain we pulled from is both very exciting and fulfilling to accomplish.